Guest editorial learning automata: theory, paradigms, and applications

L EARNING automata [1] have attracted a considerable interest in the last three decades. They are adaptive decision making devices that operate in unknown stochastic environments and progressively improve their performance via a learning process. They have been initially used by psychologists and biologists to describe the human behavior from both psychological and biological viewpoints. Learning automata have made a significant impact on all areas of engineering. They can be applied to a broad range of modeling and control problems, which are characterized by nonlinearity and a high degree of uncertainty. Learning automata have some key features, which make them applicable to a broad range of applications: they combine rapid and accurate convergence with a low computational complexity. Learning is defined as any permanent change in behavior as a result of past experience, and a learning system should therefore have the ability to improve its behavior with time, toward a final goal. In a purely mathematical context, the goal of a learning system is the optimization of a function not known explicitly [2]. Thirty years ago, Tsypkin [3] introduced a method to reduce the problem to the determination of an optimal set of parameters and then applied stochastic hill-climbing techniques. Tsetlin [4] started the work on learning automata during the same period. An alternative approach to applying stochastic hillclimbing techniques, introduced by Narendra and Viswanathan [5], is to regard the problem as one of finding an optimal action out of a set of allowable actions and to achieve this using stochastic automata. The difference between the two approaches is that the former updates the parameter space at each iteration while the latter updates the probability space. The stochastic automaton attempts a solution of the problem without any information on the optimal action. One action is selected at random, the response from the environment is observed, action probabilities are updated based on that response,